V03 issue721 - fix so filetype detection is now an optional extra #734
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…_WIP "possibly undeclared" option message should only be printed once.
These were found when developing unit tests
* New sarracenia init tests * Add pytest-mock requirement for unit tests * Sarracenia init unit test This has about 90% coverage * Update tests README with extra info on config Debugging tests wasn't working, and it's because generating code coverage while debugging breaks debugging. * Add try/except around init tests that use xattr module Apparently it's not a standard module everywhere, so this is a workaround Could also juse use SR3's filemetadata methods, as they handle things "nicely"
* Expand test debugging (pretty) Pretty printing wasn't awesome, but now it's better. * Move Options into a proper class * Move message creation into method This ensures that there's no collisions between tests if we modify properties/dict values * Use deepcopy on WorkList creation in each test This ensures that they're always unique, and have no collisions between tests * Fix broken retry tests * Add Note to top of retry "steps" test It's disabled, but can serve as good documentation of how to do it. * Capture test log existance, and assert against that It used to assert inside the "if" statement, but that would only ever be True. This way, it would be possible for the logs to be missing, and the test will fail. * Fix redisqueue cleanup test No longer need to specify a fake server, becasue we're checking the keys a little more inteligently. * Clean up commented code * Expand test debugging (pretty) Pretty printing wasn't awesome, but now it's better. * Move Options into a proper class * Move message creation into method This ensures that there's no collisions between tests if we modify properties/dict values * Use deepcopy on WorkList creation in each test This ensures that they're always unique, and have no collisions between tests * Fix broken retry tests * Add Note to top of retry "steps" test It's disabled, but can serve as good documentation of how to do it. * Capture test log existance, and assert against that It used to assert inside the "if" statement, but that would only ever be True. This way, it would be possible for the logs to be missing, and the test will fail. * Fix redisqueue cleanup test No longer need to specify a fake server, becasue we're checking the keys a little more inteligently. * Clean up commented code
Also added *all* extra to install all extras.
documenting recent changes (optional filetypes extra) and recent both languages.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As explored in #721: There is a new feature in 3.0.42, where the content-type header of each file is included in the message... so people can know the filetype before downloading. It is a mandatory field in the WMO messages, so needed the functionality to be able to export cleanly, and itś a good thing anyways, so added it to base message format.
That required an additional dependency ¨python-magic" It turns out this dependency is troublesome. The debian packaged python packege is python3-magic, the pip install package name is python-magic but that only works on linux. on windows you need to install python-magic-bin. On linux, the debian package has a dependency on libmagic1, but if you install with pip, it won´t pull in the dep... and containers seem to be missing it. So in some cases you need to install libmagic1 by hand separately. A bit of a mess.
Already have a mechanism for dealing with troublesome or fairly optional dependencies, in python they are called extras. but that only covers installation. There is also code built into sr3 to survey for the check for presence of extra library and only try to use it if it is present. so ´filetype' is now an extra feature. In this case, it just degrades to omitting the header in the messages it posts.
There is also some missing documentation in the install process, because the amqp library is now an optional extra, and nobody changed the docs... to install you need to install a message passing library as well as the package now, So added that to the documentation.
(some people want to be able to do minimal installs with mqtt, and resented having to have amqp on the machine.)